Repository navigation
Default implicit model routing to cost/auto instead of cost/balanced - #76602
Conversation
Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
|
✅ PR Specification Reviewer completed successfully! PR Specification ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ PR Code Quality Reviewer completed the code quality review.
|
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Design Decision Gate 🏗️Outcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅
|
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Ponytail Reviewer completed successfully! Ponytail ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
There was a problem hiding this comment.
Reviewed the changed lines and generated lock/golden updates. I found no actionable correctness, performance, or maintainability issues that warrant an inline comment. The change consistently switches only the implicit default from cost/balanced to cost/auto, while explicit routing and opt-out behavior remain unchanged. Review was static; tests were not run per review instructions.
🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review to run again
There was a problem hiding this comment.
🔵 Needs a closer look
The fleet-wide default change lacks the authorized branch-run evidence required by Issue #76601 acceptance case C to validate auto-mode selection, observed AIC, and task outcomes on the reviewed revision.
No open issues 🎉
What changed in this PR
Updates gh-aw’s implicit model-routing default from cost/balanced to cost/auto, addressing #76601 so eligible workflows use task-specific routing modes. Explicit choices, eligibility rules, runtime bypasses, and opt-outs remain unchanged.
Changes:
- Switches the compiler default to
autoand updates implicit-routing tests, including the batch notice assertion. - Aligns documentation and schema descriptions, explaining the existing fallback to
balanced. - Refreshes generated workflow locks and WASM golden expectations to reflect the new mode.
| File | Description |
|---|---|
pkg/workflow/testdata/TestWasmGolden_CompileFixtures/with-imports.golden |
Updates routing notice and metadata expectations. |
pkg/workflow/testdata/TestWasmGolden_CompileFixtures/smoke-copilot.golden |
Updates routing notice and metadata expectations. |
pkg/workflow/testdata/TestWasmGolden_CompileFixtures/playwright-cli-mode.golden |
Updates routing notice and metadata expectations. |
pkg/workflow/testdata/TestWasmGolden_CompileFixtures/custom-safe-output-max-length.golden |
Updates routing notice and metadata expectations. |
pkg/workflow/testdata/TestWasmGolden_CompileFixtures/basic-copilot.golden |
Updates routing notice and metadata expectations. |
pkg/workflow/testdata/TestWasmGolden_AllEngines/pi.golden |
Updates Pi routing metadata expectation. |
pkg/workflow/testdata/TestWasmGolden_AllEngines/copilot.golden |
Updates Copilot routing notice and metadata expectations. |
pkg/workflow/model_routing.go |
Updates the implicit-routing field comment. |
pkg/workflow/model_routing_test.go |
Checks the new default and batch notice. |
pkg/workflow/model_routing_default.go |
Changes the default mode constant to auto. |
pkg/workflow/engine.go |
Updates the opt-out field comment. |
pkg/workflow/compiler_mutators.go |
Updates the batch-count method comment. |
pkg/parser/schemas/main_workflow_schema.json |
Updates default-routing descriptions without changing validation. |
docs/src/content/docs/reference/model-routing.md |
Documents cost/auto and the balanced fallback. |
docs/src/content/docs/reference/glossary.md |
Updates the routing default definition. |
docs/src/content/docs/reference/frontmatter-full.md |
Updates default and opt-out comments. |
docs/src/content/docs/reference/engines/pi.md |
Documents Pi’s new implicit routing mode. |
docs/src/content/docs/reference/engines/copilot.md |
Documents Copilot’s new implicit routing mode. |
docs/src/content/docs/reference/engines/codex.md |
Documents Codex’s new implicit routing mode. |
docs/src/content/docs/reference/engines/claude.md |
Documents Claude’s new implicit routing mode. |
docs/src/content/docs/reference/engines.md |
Updates the shared default-routing description. |
.github/workflows/smoke-work-queue-worker.lock.yml |
Sets generated routing notice, metadata, and objective to auto. |
.github/workflows/smoke-work-queue-dispatcher.lock.yml |
Sets generated routing notice, metadata, and objective to auto. |
.github/workflows/smoke-security-alerts.lock.yml |
Sets generated routing notice, metadata, and objective to auto. |
.github/workflows/smoke-issues.lock.yml |
Sets generated routing notice, metadata, and objective to auto. |
.github/workflows/slide-deck-maintainer.lock.yml |
Sets generated routing notice, metadata, and objective to auto. |
.github/workflows/refiner.lock.yml |
Sets generated routing notice, metadata, and objective to auto. |
.github/workflows/plan.lock.yml |
Sets generated routing notice, metadata, and objective to auto. |
.github/workflows/objective-impact-report.lock.yml |
Sets generated routing notice, metadata, and objective to auto. |
.github/workflows/enterprise-docs-audit.lock.yml |
Sets generated routing notice, metadata, and objective to auto. |
.github/workflows/daily-team-status.lock.yml |
Sets generated routing notice, metadata, and objective to auto. |
Review effort: Balanced (auto)
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /tdd and /codebase-design. The change is consistent end to end: the defaultModelRoutingMode constant drives both notices, the schema descriptions, the golden files, and the 92 lock files. auto is already a valid mode in model_routing.go. One minor test gap is noted inline.
📋 Key Themes & Highlights
Key Themes
- Test gap: the batch-notice assertion checks the helper string, not the emitted stderr at
pkg/cli/compile_pipeline.go:916. - Scope note: the pre-fetched patch omitted the docs and lock-file changes. I reviewed the committed HEAD
d0e0548directly. The docs are updated and all 92 locks showmode auto.
Positive Highlights
- ✅ Single source of truth for the default, so the code, notice, and tests cannot drift.
- ✅ Explicit modes, opt-outs, and model-variable bypasses are untouched.
- ✅ Targeted
go test ./pkg/workflowrun for the routing default and WASM golden tests passes at HEAD.
Unverified
- The docs claim that
autofalls back tobalanced. That fallback lives in the external AWF router and is not implemented in this repo, so I could not verify it here.
@copilot please address the review comment above.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~6 AIC · ⌖ ~2 AIC · ⊞ 6.6K
Comment /matt to run again
| @@ -969,21 +969,22 @@ Report a finding. | |||
| require.NoError(t, compiled.err) | |||
| require.Zero(t, compiled.warnings, "the implicit default must not add warnings") | |||
| require.Equal(t, 1, strings.Count(compiled.stderr, defaultRoutingNoticeMarker), "one informational notice per workflow") | |||
| require.Contains(t, compiled.stderr, "goal cost and mode balanced") | |||
| require.Contains(t, compiled.stderr, "goal cost and mode auto") | |||
| require.Contains(t, DefaultModelRoutingBatchNotice(2), "goal cost and mode auto", "batch compilation reports the same default") | |||
There was a problem hiding this comment.
[/tdd] This assertion only checks the string returned by DefaultModelRoutingBatchNotice, so it would not catch a regression where the aggregated notice stops reaching stderr. The emitter at pkg/cli/compile_pipeline.go:916 has no test in pkg/cli.
💡 Suggested test
Add a batch-compile test in pkg/cli that runs two eligible workflows through the batch pipeline and asserts that stderr contains goal cost and mode auto exactly once. This pins the emitted text, not just the helper.
@copilot please address this.
|
🎉 This pull request is included in a new release. Release: |
Eligible workflows that set neither a model nor routing were routed with
goal: cost/mode: balanced. This switches the implicit default tomode: auto, so the router picks the mode for each run's task. When classification is unavailable or recommendsunknown,autofalls back tobalanced.Only the implicit default changes. Eligibility, model-family restrictions, the runtime model-variable bypass, the
engine.model-routing: falseopt-out, explicit routing (modeis still required) and main-agent-only scope are unchanged. This is the routing modeauto, notmodel: auto.Changes
Compiler:
defaultModelRoutingModeis now"auto"inpkg/workflow/model_routing_default.go. The single-workflow and batch compile notices read the same constant, so both now saymode autoand remain informational. Comments and schema descriptions that namedcost/balancedare updated.Docs: the default claim is updated in:
reference/model-routing.md: the default-routing section, two other mentions, and a new sentence on theauto→balancedfallback;reference/engines.mdandreference/engines/{copilot,claude,codex,pi}.md;reference/glossary.mdandreference/frontmatter-full.md.Examples of explicit modes are untouched.
Tests:
model_routing_test.gochange, plus a new batch-notice assertion. Explicit-mode tests are untouched.balanced→auto.Locks (
make recompile): 92 lock files change, matching the 92 locks on main that containid: resolve-model-routing. Each changed line is onlybalanced→auto, in three places:echo "Model routing enabled: goal cost, mode auto"apiProxy.routing.objective={"goal":"cost","mode":"auto"}GH_AW_INFO_MODEL_CONFIGNo other lock content changes (no hash, version or pin drift). No explicitly routed or opted-out lock changes. For the provenance-managed workflows (
agentic-token-audit,agentic-token-optimizer), the source Markdown is untouched and only the locks are regenerated.Changed locks (92)
agent-performance-analyzer, agentic-token-audit, agentic-token-optimizer, architecture-guardian, auditor, aw-issue-clustering, breaking-change-checker, ci-coach, cli-consistency-checker, copilot-opt, copilot-pr-merged-report, copilot-pr-nlp-analysis, copilot-pr-prompt-analysis, craft, daily-agent-of-the-day-blog-writer, daily-ambient-context-optimizer, daily-assign-issue-to-user, daily-compiler-quality, daily-compiler-threat-spec-optimizer, daily-declarative-engine-improver, daily-ecosystem-explorer, daily-experiment-report, daily-formal-spec-verifier, daily-geo-optimizer, daily-graft-intelligence, daily-pr-review-cursor, daily-regression-audit-kiro, daily-report-dispatcher, daily-safe-output-integrator, daily-schema-audit-cursor, daily-secrets-analysis, daily-security-observability, daily-spdd-spec-planner, daily-spec-coverage-kiro, daily-squid-image-scan, daily-syntax-error-quality, daily-team-status, daily-testify-uber-super-expert, daily-trajectory-grader-implementer, daily-work-queue-formal-verification, daily-work-queues-report, daily-workflow-security-model, daily-workflow-updater, dataflow-pr-discussion-dataset, dead-code-remover, delight, deployment-incident-monitor, dev-hawk, dictation-prompt, docs-noob-tester, draft-pr-cleanup, engine-conformance-copilot, engine-conformance-dispatcher, enterprise-docs-audit, eslint-factory-dispatcher, eslint-miner, firewall-escape, firewall, layout-spec-maintainer, mcp-inspector, mergefest, objective-impact-report, org-health-report, plan, pr-finisher, pr-nitpick-reviewer, refiner, release, repository-quality-improver, security-compliance, slide-deck-maintainer, smoke-checkout-pr-dispatch, smoke-copilot-dynamic-workflow, smoke-create-cross-repo-pr, smoke-issues, smoke-multi-pr, smoke-project, smoke-security-alerts, smoke-update-cross-repo-pr, smoke-work-queue-dispatcher, smoke-work-queue-worker, smoke-workflow-call-with-inputs, spec-extractor, spec-librarian, stale-pr-cleanup, tidy, ubuntu-image-analyzer, visual-regression-checker, weekly-blog-post-writer, weekly-editors-health-check, workflow-health-manager, workflow-skill-extractor
Notes for reviewers
pkg/linters,TestDocSurfacesMatchRegistryAndSpecListandTestDocGo_AnalyzersMatchREADMEalso fail on the base commit.pkg/cli,TestDownloadWorkflowLogsForTargets{ConcurrentAndResilient,UsesOneWallClockTimeout}hit their time limits during the full test run. They pass when run alone with this change.